1. Field of the Invention
The present invention relates to test techniques and test apparatus for remote autonomous validation of wireless app performance. More specifically, the subject matter relates to methods, systems, and apparatus for remote, autonomous testing and/or measuring of overall app service accessibility, availability and related performance.
2. Description of Related Art
Wireless network subscribers access online services using apps on their devices (smartphones, tablets, PCs) and network subscriber terminal equipment. This network subscriber terminal equipment includes cell sites and smaller wireless access points (APs), with related network routers and network switches that are further connected to an external backhaul transport mechanism such as copper line, microwave link, Wi-Fi, WiMax, cable, optical fiber or other transmission medium.
Various categories of app-based communication are possible through these packet-based networks, including data, video and voice. For example, voice over Internet protocol (VoIP) apps such as Skype, enable voice to be carried over wireless networks using Internet protocol (IP) communications. Video streaming apps, such as YouTube, allow digitized video imagery to be transmitted using these same wireless networks. Social media networks, such as Twitter and Facebook, allow files, images and messages to be exchanged between groups of connected people. Additionally file transfer and backup apps, such as DropBox, allow further app specific wireless transmissions on this same basis.
It should be noted that the term ‘app’ as used herein refers to a piece of software (an ‘application’) designed to fulfill a particular purpose, as opposed to system software. Such a program may, for example, be downloaded by a user to a mobile device. However, the invention is not intended to be limited in regard to the type or nature of the app being tested, the type of device on which it may be executed by users, or the method of installation.
While wireless service providers strive to avoid problems that affect subscribers' app performance, problems still often arise. This can be a problem for organizations where performance problems affect app users at their site. For example, if guests experience difficulties in accessing or using popular apps such as Facebook, YouTube, Twitter etc at a particular hotel, this may dissuade those app users from re-visiting that hotel in the future.
When app performance problems arise at a given location, the ability to remotely diagnose and validate those problems has been limited to date. Various centralized tools exist in the art for passive diagnosis and validation of subscriber performance by analyzing information from the network transmissions, or transmitted from the subscriber's own device, such as location and signal quality. A determination on how to proceed with correcting the issue may be made based on the list of diagnostic results that are available but, by their nature, these are only very general assessments of the actual app service delivery. Therefore, decisions and strategies for addressing the problem will be made using information that is not sufficiently detailed or focused, leading to ineffective solutions.
Additionally, many complaints are received from subscribers about quality issues relating to a specific app but these issues are not available or visible to the network operator from the packet-layer network information available.
Currently, in order to ensure that app service is accessible and performing from the subscriber perspective, it is considered best practice to send an engineer to visit a site or location with recurring sub-par app performance reports, validate whether or not a fault or faults actually exists and determine where the fault lies. This process is time-consuming and costly. In such cases, various RF test instruments such as spectrum analyzers, vector signal analyzers, vector signal generators and RF power meters are used to test these remote environments. Parameters of the test instruments and wireless network devices under test are manually set by operators so as to manually perform RF measurements and record network-layer data.
Furthermore, other aspects of the environment and set-up, such as network components and access lines must be further isolated and validated without the localized environmental element of the on-site equipment. Then, all of this data is analyzed manually to identify the root cause or causes of the degraded app performance to generate a service validation report—such a process is time-consuming, susceptible to errors and expensive. Additionally, app performance variations are transient by time-of-day or day-of-week or month or based on the number of users in a particular location at any one time and cannot be solved other than having an engineer on site 24×7 to determine such fluctuation in demand.
Therefore, there is a need to validate app performance visibility by locale for a network service, on an ongoing basis, so as to overcome the above-described drawbacks. Such an improved solution has now been devised.